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When  U.S.  Government  drawings,  specifications,  or  other  data  are  used 
for  any  purpose  other  than  a definitely  related  Government 
procurement  operation,  the  Government  thereby  incurs  no 
responsibility  nor  any  obligation  whatsoever,  and  the  fact  that  the 
Government  may  have  formulated,  furnished,  or  in  any  way  supplied 
the  said  drawings,  specifications,  or  other  data  is  not  to  be  regarded  by 
implication  or  otherwise,  as  in  any  manner  licensing  the  holder  or  any 
other  person  or  corporation,  or  conveying  any  rights  or  permission  to 
manufacture,  use,  or  sell  any  patented  invention  that  may  in  any  way 
be  related  thereto. 
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NTIS,  it  will  be  available  to  the  general  public,  including  foreign 
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— simple  and  accurate  process  is  needed  for  comparing  proposed  equipment  to  existing  operational  equipment 
in  order  to  forecast  the  resources  required  to  support  the  proposed  equipment.  Maintenance  manpower  modeling 
needs  such  a process  to  predict  manpower,  spares,  and  support  equipment  of  newly  designed  weapon  systems. 

As  a first  step  toward  developing  the  needed  techniques  fot  comparing  old  and  new  equipment,  a systematic 
study  was  made  of  the  present  state-of4he-art.  Current  literature  was  reviewed,  personnel  who  perform 
comparability  analysis  were  interviewed,  and  past  comparablity  studies  were  analyzed.  This  effort  found  that  a 
distinction  dtould  be  made  between  the  broad  topic  of  “comparability  studies"  and  the  specific  procedures  of 
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Item  20  (Continued) 

^‘comparability  analysis.”  A comparability  study  is  the  overall  process  used  to  estimate  resource  requirements  for 
new  equipment  using  the  procedures  of  comparability  analysis  to  find  operational  equipment  which  is  similar  to  the 
proposed  equipment. 

It  also  was  found  that  no  standardized,  reliable  procedure  exists  for  performing  comparability  studies 
including  comparability  analysis.  It  is  recommended  that  a quantitative  procedure  be  developed  for  comparing  new 
equipment  with  equipment  in  the  operational  inventory.  It  is  further  recommended  that  a quantitative  procedure  be 
developed  for  computing  an  adjustment  factor  relating  the  operational  equipment  to  the  proposed  equipment.  A 


SUMMARY 


Problem 

An  important  step  in  the  early  estimation  of  manpower  requirements  for  advanced  weapon  systems  is 
an  accurate  “comparability  study.”  A comparability  study  is  the  overall  process  used  to  estimate  resource 
requirements  for  newly  designed  weapon  systems  by  basing  the  estimate  on  the  resources  used  by 
operational  equipment  that  is  similar  to  that  of  the  proposed  weapon  system.  The  comparability  study 
includes  a “comparability  analysis,”  which  is  the  procedure  followed  to  identify  operational  equipment 
similar  to  the  proposed  equipment.  Simple  yet  accurate  procedures  are  needed  for  performing  both 
comparability  studies  and  comparability  analyses. 

Purpose  of  the  Study 

The  purpose  of  this  investigation  was  to  review  the  state-of-the-art  regarding  the  procedures  for 
performing  a comparability  study  including  comparability  analysis.  This  effort  had  four  principal 
objectives:  (a)  to  review  the  printed  guidelines  available,  (b)  to  determine  the  actual  process  in  use,  (c)  to 
determine  the  comparability  analysis  methods  in  use,  and  (d)  to  determine  the  problems  associated  with 
comparability  studies. 

Approach 

A review  was  made  of  the  available  literature  and  guidance  to  determine  the  recommended 
comparability  study  process.  Then,  material  was  reviewed  on  the  Air  Force  comparability  studies 
performed  for  the  A-10,  B-l,  F-16,  and  YC-15  aircraft  and  the  PLSS  (Precision  Location  Strike  System). 
Material  on  the  failure  rates  of  aircraft  systems  was  also  reviewed  to  determine  the  comparability 
techniques  used.  Finally,  project  engineers  and  administrators  of  comparability  studies  were  interviewed  to 
determine  their  involvement  in  the  process  and  the  problems  they  encountered. 

Results 

The  four  objectives  of  this  study  have  been  accomplished.  First,  the  literature  and  guidance  material 
were  found  to  be  very  limited  in  scope.  Second,  the  actual  process  used  by  engineers  varied  greatly  from  the 
limited  guidance.  Third,  the  methods  used  by  the  engineers  were  found  to  be  undefined  and  not 
systematically  structured.  Finally,  many  of  the  problems  arise  from  this  lack  of  guidance  and  structure, 
resulting  in  confusion  and  doubts  about  the  accuracy  of  the  results. 

Recommendations 

It  is  recommended  that  a quantitative  comparability  analysis  technique  using  such  methods  as  scales, 
matrices,  and  regression  analysis  be  developed.  It  also  is  recommended  that  a quantitative  method,  or 
technique,  be  developed  for  computing  a data  adjustment  factor.  A third  recommendation  is  the 
development  of  a technique  for  determining  the  Mean  Sorties  Between  Maintenance  Actions  for  new 
equipment  when  comparable  old  equipment  cannot  be  found. 
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PREFACE 


This  research  effort  was  supported  by  the  Air  Force  Human  Resources  Laboratory 
(AFHRL)  under  Work  Unit  ASDS  2036,  Comparability  Analysis  of  Newly  Designed 
Weapon  Systems.  This  effort  was  aided  by  the  insight  of  Captain  Larry  Howell  and 
Captain  Albert  Wong  of  the  Engineering  Specialties  Division,  Modeling  and  Analysis 
Branch  (Aeronautical  Systems  Division  of  Air  Force  Systems  Command), 
Wright-Patterson  AFB,  Ohio.  The  advice  and  guidance  of  Dr.  William  Askren,  Mr.  Robert 
Deem,  and  Mr.  Frank  Maher  of  AFHRL  at  Wright-Patterson  AFB  were  the  main 
directional  force  of  this  study.  Appreciation  is  extended  to  the  numerous  engineers  in  the 
various  system  program  offices  working  at  Wright-Patterson  AFB  who  worked  on 
comparability  analysis  studies  and  to  all  of  the  individuals  in  the  Modeling  and  Analysis 
Branch  for  their  assistance  and  participation  in  the  data  collection  phase. 
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EQUIPMENT  COMPARABILITY  TECHNIQUES 
USED  DURING  EARLY  SYSTEM  DESIGN 

1.  INTRODUCTION 


Problem 

An  important  step  in  the  early  estimation  of  manpower  and  other  resources  requirements  for 
advanced  weapons  systems  is  an  accurate  comparability  study.  A comparability  study  is  the  overall  process 
used  to  develop  data  on  newly  designed  weapons  systems  by  (a)  selecting  operational  equipment  that  is 
similar  to  that  of  the  proposed  weapon  system  and  (b)  adjusting  the  resource  data  associated  with 
operational  equipment  to  reflect  the  unique  characteristics  of  the  proposed  equipment. 

The  comparability  study  includes  developing  maintenance  demand  rates  for  the  proposed  equipment 
which,  in  turn,  can  be  used  to  determine  resource  requirements,  such  as  spares,  manpower,  and  support 
equipment  for  the  weapon  system.  The  comparability  study  also  includes  a comparability  analysis,  a 
systematic  procedure  for  finding  operational  equipment  that  is  similar  to  the  proposed  equipment. 

A simple  and  accurate  procedure  is  needed  for  performing  comparability  analysis.  However, 
communications  with  systems  engineering  personnel1  performing  comparability  studies,  including  the 
sub-effort  comparability  analysis,  indicate  that  a simple  and  accurate  procedure  does  not  exist. 

Purpose  of  the  Study 

The  purpose  of  this  investigation  was  to  review  the  state-of-the-art  regarding  the  procedures  for 
performing  a comparability  study,  including  comparability  analysis,  and  to  define  the  areas  of  needed 
research  for  improving  the  procedures.  This  effort  had  four  principal  objectives:  (a)  to  review  the  printed 
guidelines  available  for  accomplishing  comparability  studies,  (b)  to  determine  the  actual  process  used  by  the 
administrators’  of  comparability  studies,  (c)  tu  determine  the  various  comparability  analysis  methods  used 
by  the  engineers’,  and  (d)  to  determine  the  problems  associated  with  performing  comparability  studies. 


II.  APPROACH 

First,  a review  was  made  of  the  literature  and  guidance  available  to  determine  the  recommended 
comparability  study  process.  Second,  material  was  reviewed  on  the  Air  Force  comparability  studies 
performed  for  the  A-IO.  B-l,  F-16,  and  YC-15  aircraft  and  the  PLSS  (Precision  Location  Strike  System), 
and  material  was  reviewed  on  the  failure  rates  of  aircraft  systems  by  Shurman  (Note  1)  in  order  to 
determine  comparability  techniques  used.  Finally,  139  project  engineers  and  four  administrators  of 
comparability  studies  were  interviewed  to  determine  their  involvement  in  the  process  and  the  problems  that 
they  encountered.  Of  the  engineers  interviewed,  21  were  with  the  A-10, 34  with  the  B-l,  36  with  the  F-16, 
and  48  associated  with  the  YC-1 5. 


111.  RESULTS 


Review  of  Printed  Guidelines 

The  work  of  Tetmeyer  (Note  2)  is  the  only  major  guidance  available  on  doing  an  equipment 
comparability  study.  A Boeing  study,  Shurman  (Note  1),  gave  several  unique  methods  for  analyzing  data 
which  could  be  used  to  perform  a comparability  analysis  of  a specific  subsystem.  A combination  of  the 


1 Aeronautical  Systems  Division,  Modeling  and  Analysis  Branch  ( ASD/ENESA). 

1 An  “administrator"  for  the  purpose  of  this  report  is  the  individual  who  oversees  the  overall  comparability  study. 
’The  term  "engineer”  is  restricted  to  the  project  engineer  who  performs  the  comparability  analysis. 


information  in  these  two  sources  provides  the  genera]  guidelines  for  accomplishing  a comparability  study 
and  is  described  in  the  following  paragraphs.  However,  no  step-by-step  process  has  been  developed  to  aid 
engineers  or  administrators. 

The  approach  recommended  by  Tetmeyer  (Note  2)  is  to  identify  a comparable  piece  of  equipment 
that  is  already  in  use  for  a similar  purpose  in  a similar  physical  and  operational  environment  and  to  use  the 
field  experience  for  it  as  a baseline  for  predicting  maintenance  frequency  on  the  new  equipment.  This 
assumes  that  many  of  the  unmeasurable  factors  will  affect  both  items  in  a similar  way  and  that  any  design 
deficiencies  on  the  new  equipment  w31  be  corrected  during  testing. 

Every  three-digit-level  work  unit  code  (WUC)  on  the  new  system  should  be  paired  with  the  same 
three-digit  WUC  on  existing  systems.  This  is  because  troubleshooting,  functional  checks,  and  adjustments 
cannot  duplicate  work,  and  many  minor  repair  jobs  usually  are  reported  at  this  level.  Within  each 
subsystem,  comparable  items  also  would  be  identified  for  significant  Line  Replaceable  Units  (LRUs)  at  the 
four-  or  five-digit  level.  However,  this  level  of  information  and  configuration  detail  on  the  LRUs  for  the 
new  equipment  normally  is  not  defined  until  the  aircraft  has  completed  design  reviews  during  the  full  scale 
development  phase. 

Tetmeyer  (Note  2)  also  designed  a sample  sheet  which  structured  the  data  required  from  the 
engineers.  The  administrators  only  had  to  identify  which  subsystem  a specific  engineer  was  to  analyze  and 
provide  him  with  the  sample  sheet.  Required  data  included  the  comparable  WUCs,  the  aircraft  in  which  the 
subsystem  was  located,  the  adjustment  factor,  and  the  rationale  in  determining  the  adjustment  factor. 

There  is  seldom  a perfect  match  between  two  pieces  of  equipment  on  all  characteristics  that  could 
impact  frequency  of  corrective  maintenance.  The  mathematical  methods  described  by  Shurman  (Note  1) 
can  be  used  to  help  determine  which  equipment  is  comparable  and  the  level  of  comparability.  These 
methods  give  a statistical  figure  with  which  to  adjust  the  data  on  the  operational  equipment  identified. 
However,  a few  subsystems  will  be  so  different,  or  incorporate  such  new  state-of-the-art  components,  that 
no  comparable  equipment  can  be  identified.  In  these  cases,  the  estimated  maintenance  frequency  must  be 
built  up  or  factored  from  reliability  demonstration  data. 

Actual  Process  Being  Used  in  Comparabiity  Studies 

The  following  paragraphs  and  Figure  1 describe  how  a comparability  study  actually  is  being 
accomplished.  First,  the  System  Program  Office  (SPO)  for  the  weapon  system  requests  manpower  estimates 
from  the  Logistics  Composite  Model  (LCOM)  Branch.  Second,  the  LCOM  Branch  determines  the  need  for  a 
comparability  analysis,  and  the  administrator  of  the  comparability  study  requests  a list  of  proposed 
subsystems  and  WUCs  for  the  total  aircraft  from  the  SPO.  The  third  step  is  the  preparation  of  the  list  of 
subsystems  and  WUCs  for  all  equipment  on  the  proposed  aircraft  by  the  SPO  preliminary  designers.  Then 
the  administrator  of  the  comparability  study  in  Step  4 reviews  the  lists  for  completeness  and  eliminates  any 
overlapping  WUCs.  Steps  3 and  4 normally  are  accomplished  simultaneously  and  interactively. 

Next,  the  administrator  divides  the  list  into  specific  subsystems  and  decides  who  will  determine  the 
comparable  equipment  for  each  subsystem.  Some  administrators,  Step  5A,  choose  to  determine  comparable 
equipment  for  some  of  the  subsystems  because  of  their  knowledge  of  the  equipment  and  the  operational 
scenario.  However,  most  of  the  subsystems  are  given  to  the  SPO  engineers  to  determine  what  is  comparable, 
Step  SB. 

During  Step  SB,  these  requests  and  an  instructional  handout  for  accomplishing  comparability  analysis 
are  delivered  to  the  individual  SPO  engineers.  Any  problems  or  questions  by  the  engineers  during  this  phase 
are  directly  referred  to  the  administrator  of  the  study.  The  administrator  is  kept  busy  fulltime  delivering 
and  collecting  questionnaires,  answering  inquiries,  and  explaining  requirements.  However,  later  in  Step  S of 
the  development  of  a comparability  study,  the  administrator  can  start  managing  the  data  already  provided 
by  tome  of  the  engineers. 

After  receiving  the  request  to  investigate  the  comparability  of  specific  subsystems  of  LRUs  during 
Step  SB,  the  SPO  engineer  determines  what  operational  aircraft  and  equipment  are  comparable  to  the 
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Figure  I.  Flow  chart  of  comparability  studies. 
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equipment  on  the  proposed  aircraft.  As  a part  of  the  comparability  analysis,  a reliability  or  adjustment 
factor  is  developed  by  the  engineers.  For  example,  if  the  comparable  item  is  exactly  the  same,  the 
adjustment  factor  should  be  1.0.  An  adjustment  factor  of  2.0  indicates  that  the  item  of  the  proposed 
aircraft  fails  twice  as  often  and  is  less  reliable,  while  an  adjustment  factor  of  .5  indicates  that  the  proposed 
item  fails  half  as  often  and  is  more  reliable.  During  this  phase,  the  determination  of  what  data  apply  is  at 
the  discretion  of  the  engineer.  However,  most  engineers  are  not  aware  of  the  total  Air  Force  inventory  from 
which  to  choose,  lienee  inadequate  comparisons  occasionally  are  made. 

In  Step  6 the  administrator  of  the  comparability  study  compiles  the  comparability  data  and 
adjustment  factors  and  insures  that  all  equipment  and  subsystems  have  been  analyzed  and  data  corrected. 
During  Step  7,  the  administrator  of  the  study  obtains  maintenance  data  (e.g..  Maintenance  Manhours  per 
"Flying  Hours)  for  the  operational  aircraft  from  the  computer  tapes  of  the  Air  Force  Maintenance  Data 
Collection  System  (Air  Force  Form  66-1  Data)  for  the  previous  6 months  to  a year.  The  administrator  then 
converts  the  maintenance  manhours  per  flying  hours  to  mean  sorties  between  maintenance  actions 
(MSBMA).  The  MSBMA  data  developed  on  each  piece  of  operational  equipment  are  multiplied  by  the 
adjustment  factor  to  determine  the  MSBMA  for  the  proposed  equipment.  The  MSBMA  data  for  each 
subsystem  are  then  combined  to  provide  an  MSBMA  for  the  total  aircraft  and  used  to  determine  the 
manpower  requirements,  which  are  delivered  to  the  SPO. 

The  following  differences  were  found  between  the  actual  process  described  above  and  the  process 
recommended  by  Tetmeyer  (Note  2)  for  accomplishing  a comparability  study.  Tetmeyer  assumed  that  the 
breakdown  into  subsystems  and  WUCs  already  had  been  accomplished.  However,  in  actuality,  a listing  of 
subsystems  and  WUCs  has  to  be  prepared.  Also,  Tetmeyer  suggested  that  the  proposed  subsystem  be  built 
up  from  comparable  LRUs  when  there  is  not  a sufficiently  comparable  subsystem.  In  the  actual  process,  an 
MSBMA  is  developed  for  the  proposed  subsystem  without  regard  to  comparable  LRUs.  Finally,  Tetmeyer 
recommended  that  the  total  weapon  system  be  analyzed  to  provide  the  most  complete  data  for  manpower 
analysis.  However,  in  several  studies  the  data  for  a number  of  significant  WUCs  were  omitted  when  no 
operational  equipment  could  be  identified  as  comparable. 

Comparability  Analysis  Methods  Used  by  the  Engineers 

Comparability  analysis  is  a systematic  process  for  finding  operational  equipment  that  is  similar  to 
proposed  equipment.  Comparability  analysis  usually  is  accomplished  at  the  subsystem  level  or  three-digit 
WUC,  and  the  computations  are  combined  to  establish  an  overall  numerical  index  to  adjust  the  historical 
maintenance  data  available.  Each  subsystem  usually  is  analyzed  by  a different  engineer;  this  can,  and 
usually  does,  cause  the  end  product  to  be  derived  by  various  methods.  The  available  methods  represent  a 
continuum,  ranging  from  those  methods  that  are  largely  intuitive  (expert  estimate)  through  those  that  are 
explicit  and  objective  (modeling).  The  methods  described  below  are  used  to  determine  both  the  comparable 
piece  of  equipment  or  aircraft  and  the  adjustment  factor.  Separate  methods  possibly  could  be  used  to 
determine  first  the  comparable  equipment  and  then  the  adjustment  factor. 

A.  Expert  Estimate.  The  expert  estimate  method  is  a procedure  in  which  the  engineer  uses  Iris  own 
intuitive  skills  and  background  to  determine  the  comparable  equipment  and  the  adjustment  factor.  There  is 
minimal  reliance  guides  and  other  formal  data.  Estimation  proceeds  by  a combination  of  system  familiarity, 
experience,  and  intuition.  Hopefully,  engineers  seldom  rely  expert  estimates  alone;  instead,  they  use  a 
combination  of  methods.  Some  design  engineers  simply  review  the  design  concepts  for  the  new  system, 
look  for  familiar  subsystems,  and  add  a blanket  percentage  increase/decrease  as  an  adjustment  factor  for 
newness.  Other  agencies  attempt  to  increase  the  reliability  of  their  estimates  by  convening  a group  of 
experts  and  having  them  arrive  at  predictions  through  discussions.  An  important  feature  of  the  expert 
estimate  method  is  that  judgements  can  be  made  on  unusual  factors,  such  as  environmental  considerations. 

B.  Historical  Comparison.  The  .historical  comparison  method  relates  proposed  equipment  to  similar 
operational  equipment  and  uses  the  past  experience  of  operational  equipment  to  determine  the  resource 
requirements  of  the  proposed  equipment.  Complete  data  are  collected  on  the  operational  equipment  and 
are  kept  on  computer  tape  with  programs  available  for  data  retrieval  and  system  matching.  Predictions  for  a 
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new  system  are  made  by  considering  the  new  system  as  a group  of  subsystems  similar  to  those  already  in 
existence.  The  comparability  data  for  the  known  subsystems  are  retrieved  mechanically  and  serve  as  the 
basis  for  the  predictions  of  the  new  subsystem.  An  adjustment  to  the  prediction  (increasing/decreasing)  is 
made  by  expert  estimates  and/or  abbreviated  forms  of  the  dominant  factors  and  modeling  methods.  Data 
kept  in  the  data  banks  are  upgraded  consistently  as  the  actual  information  comes  in  for  systems  in  the  Air 
Force  inventory. 

C.  Dominant  Factors  (High  Drivers).  The  dominant  factor  technique  of  the  estimation  of 
comparability  involves  complex  manipulation  of  hundreds  of  variables  affecting  equipment.  A dominant 
factor,  or  high  driver,  (such  as  landing  weight)  is  a single  variable  that  stands  ahead  of  a whole  group  of 
variables  (such  as  steering,  control,  tires)  and  is  critical  to  determining  the  comparability  of  two  different 
landing  gears.  If  an  operational  definition  and  early  measurement  of  the  dominant  factor  are  possible,  it  will 
not  be  necessary  to  measure  and  interrelate  the  numerous  variables  represented  by  other  factors. 

Dominant  concepts  are  sought  to  avoid  the  difficult  task  of  accounting  for  multiple  independent 
variables.  The  peril  is  that  some  revolutionary  change  in  design  or  use  of  a weapon  system  will  affect 
significantly  the  relationship  of  important  variables  on  the  dominant  factor,  thus  destroying  its  utility  as  an 
index  of  comparability.  Because  of  this  risk,  no  engineer  uses  this  method  exclusively  but  backs  up  the 
estimates  with  other  methods  (usually  some  variant  of  the  historical  comparison). 

D.  Modeling.  Modeling  is  a general  category  for  all  methods  typified  by  their  highly  objective  and 
explicit  nature.  Some  of  the  procedures  are  graphic  layouts,  function-logic  layouts,  formula-review 
programs,  and  predictive  math  models  (linear  and  dynamic  programming,  multiple  regression  expressions, 
etc).  The  methods  specify  in  detail  what  data  are  necessary  for  use  in  the  estimation  technique,  how  the 
data  are  to  be  used  in  the  model,  and  how  to  interpret  the  results.  Judgements  is  not  involved.  The  data 
goes  into  the  model  unaltered.  Consequently,  these  methods  have  the  advantage  of  being  checked  by 
different  people  running  the  same  data  through  again  and  comparing  the  outcome.  Modeling  techniques 
could  vary  from  simple  rating  scales  to  elaborate  simulated  systems  which  can  be  experimentally 
manipulated.  Maintenance  Manpower  Modeling  (MMM)  and  the  LCOM  are  excellent  examples  of  the 
usefulness  and  flexibility  of  modeling  techniques. 

Trend  Toward  Objectivity 

The  review  of  the  comparability  analyses  performed  by  the  engineers,  as  shown  by  Figure  2,  revealed 
that  the  users  of  comparability  analysis  seem  to  be  moving  toward  the  objective  end  of  the  methods 
continuum.  Older  comparability  studies  (such  as  for  the  A-10  aircraft)  relied  more  on  intuitive  methods 
(expert  estimate  38%)  and  less  on  objective  methods  (dominant  factors  and  modeling  10%).  Newer  studies 
(such  as  for  the  YC-14  aircraft)  relied  more  on  objective  approaches  (dominant  factors  and  modeling  54%). 
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Figure  2.  Methods  used  by  various  SPO  engineers 


Pro  Mem  Areas  Associated  with  Comparability  Analyses  and  Studies 

A number  of  problems  are  associated  with  the  comparability  analysis.  First,  there  is  no  standard 
comparability  analysis  method  that  is  relatively  accurate  in  the  early  stage  of  weapon  system  development. 
Second,  a prediction  often  is  made  in  a form  that  does  not  lend  itself  readily  to  making  design  changes. 
Third,  the  subjective  element  in  the  estimation  occasionally  is  so  large  that  the  designers  seem  reluctant  to 
accept  the  data  and  to  make  the  important  design  changes.  This  is  especially  true  when  the  method  is  of  the 
“expert  estimator"  variety. 

Furthermore,  these  problems  associated  with  the  comparability  analysis  have  created  additional 
problems  with  the  overall  comparability  study.  When  a comparability  study  is  being  developed  confusion 
arises  as  to  when  things  should  happen  and  in  what  sequence:  for  example:  when  is  it  best  to  initiate  a 
comparability  study,  when  should  subsystems  be  listed,  when  should  engineers  develop  the  adjustment 
factors,  and  when  should  an  update  be  done.  Also,  different  subsystems  have  different  significant 
parameters;  hence,  different  comparability  processes  need  to  be  developed  specifically  for  each  subsystem. 
Finally,  the  use  of  adjustment  factors  to  account  for  failure  rate  variations  between  new  and  old  equipment 
are  sometimes  objectionable,  because  the  adjustment  values  generally  are  not  known  to  an  acceptable 
degree  of  confidence. 

To  help  alleviate  some  of  these  problems,  two  separate  techniques  need  to  be  developed.  First,  a 
standard,  reliable  technique  is  required  for  comparing  equipment  currently  under  design  with  equipment 
that  is  operational  in  the  field  to  determine  the  operational  equipment  which  is  “most  similar”  to  the  newly 
designed  equipment.  Second,  a standard,  reliable  technique  is  needed  for  computing  the  adjustment  factor, 
which  is  used  to  modify  the  failure  rate  of  old  equipment  to  represent  this  failure  rate  of  the  new 
equipment. 


IV.  CONCLUSIONS  AND  RECOMMENDATIONS 

The  four  objectives  of  this  study  have  been  accomplished  and  the  findings  are  as  follows.  First,  the 
literature  and  guidance  regarding  comparability  analyses  and  studies  are  very  limited.  Second,  the  actual 
process  used  varies  greatly  from  the  guidance.  Third,  the  methods  used  by  the  engineers  were  undefined  and 
not  systematically  structured.  Finally,  many  problems  result  from  this  lack  of  structure  and  guidance, 
causing  confusion  and  doubts  about  the  accuracy  of  the  results.  The  following  research  studies  are 
recommended  in  order  to  help  structure  the  process  and  to  provide  more  reliable  and  accurate 
comparability  data: 

1.  Develop  a systematic,  quantitative,  parametric  comparability  analysis  technique  using  such 
methods  as  scales,  matrices,  and  regression  analysis. 

2.  Develop  a systematic,  quantitative  method  for  computing  the  adjustment  factor,  and  when 
subsystems  are  not  sufficiently  comparable,  develop  a technique  for  estimating  the  MSBMA  for  the  new 
equipment. 

3.  Perform  a time  analysis  of  the  comparability  study  flow  of  events  in  order  to  determine  what 
needs  to  be  done  during  a comparability  study  and  at  what  times.  Also,  as  a part  of  this  study,  the  value  of 
periodic  updates  of  the  comparability  studies  should  be  considered. 
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